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ART-UNIT: 335 

PRIMARY- EXAMINER: Smith; Ruth S. 

ATTY- AGENT -FIRM: Peacock; Deborah A. Myers; Jeffrey D. Duggan; Donovan F. 
ABSTRACT : 



The disclosure is directed to a method and apparatus for presenting a plurality of 
scanning images in a video presentation. The disclosure is particularly directed to the 
use .of such an apparatus and method in stereotactic surgery, wherein acquisition of a 
plurality of scanner images, conversion of these images into a selected standard 
format, storage of the acquired and converted images, selective recall and display of 
at least two of the images, and independent manipulation of each of the images for 
comparison are provided. Probe placement can be simulated and the images can be 
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enhanced in various ways. The images used can be, for example, CT, PET, X-ray, DSA, 
isotope, and NMR scans. Brain map atlas images can be superimposed on and fitted to the 
scan images of a particular brain. Video displays of three dimensional simulations can 
also be produced. 

109 Claims, 12 Drawing figures 
Exemplary Claim Number: 1 
Number of Drawing Sheets : 8 

BRIEF SUMMARY: 



A portion of the disclosure of this patent document contains material which is subject 
to copyright protection. The owner has no objection to the facsimile reproduction by 
anyone of the patent document or the patent disclosure, as it appears in the Patent and 
Trademark Office patent file or records, but otherwise reserves all copyright rights 
whatsoever . 

Computer program listings comprising sequences of instructions, routines, and other 
contents are provided in the microfiche appendix, which is incorporated herein by 
reference. The microfiche appendix consists of 8 sheets of microfiche, each sheet 
having 65 frames, not all of which have been used. 

1. FIELD OF THE INVENTION 

The invention relates to video presentation and more particularly to a method and 
apparatus for generating a video presentation from a variety of scanner imaging 
sources, in particular for stereotactic surgery. 

2 . BACKGROUND OF THE INVENTION 

Stereotactic surgical techniques allow physiological exploration and/or destruction of 
deep cerebral or spinal cord structures which are invisible from the surface, but which 
location can be determined by a knowledge of their coordinates in space relative to 
known anatomical and topographical landmarks. The use of stereotaxis in neurosurgical 
techniques seeks to avoid open operative approaches to these areas with a minimum of 
disturbance to surrounding structures. The technique generally involves the placement 
of fine electrodes or probes in strategic "target areas" which may comprise specific 
functional anatomical sites or morphological lesions or abnormalities. One of the major 
difficulties of stereotactic surgery is graphic conceptualization of the location of 
surgical probes inserted into deep brain structures. Not only is the probe out of the 
surgeon's sight, but it is tilted, rotated, and extended in many different directions, 
which circumstance makes it almost impossible for the surgeon to maintain a mental 
picture of the location of the probe in the brain core. The surgeon must imagine the 
location of the probe while taking into account the forward and lateral angles of the 
probe, the distance of the probe from the target, the direction that the electrode 
extends from the probe, and many other angular variables. Furthermore, the coordinate 
system of the stereotactic frame seldom corresponds to the "brain coordinate system," 
causing greater margin of error and difficulty in placement of the probe. Stereotactic 
surgery, therefore, is essentially a "blind" surgical procedure with many complex 
geometric variables. Any system which will enhance the surgeon's conceptualization of 
the procedure will greatly enhance its efficacy. 

The introduction of stereotaxis to the armamentarium of human neurosurgical technique 
has been an important addition. This is evident by its use in the treatment of many 
neurological disorders. This technique has expanded from its earlier use, primarily in 
the treatment of dyskinesias and pain syndromes, to include the treatment of seizure 
disorders, aneurysms, brain tumors and many other neuropathological conditions. In 
recent years there has been a significant increase in the number and use of 
stereotactic surgical techniques. This has been brought about by the development of new 
imaging technologies, for example, computerized axial tomography (CT) , nuclear magnetic 
resonance (NMR) scanning, various radioisotope scanning techniques, and digital 
subtraction angiography (DSA) . These imaging techniques allow the surgeon to "see" 
certain brain structures and to use these imaging technologies to aid in planning 
stereotactic surgical procedures. 

Computed tomography (CT) is well established as a valuable diagnostic and investigative 
imaging device and has revolutionized the evaluation and treatment of neurological 
conditions. Applications and use of CT technology with stereotactic and functional 
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neurosurgery are increasing. Advancing computer technology has been the basis upon 
which CT scanning technology has developed; this same technology is supporting the 
development of the newer digital subtraction angiographic (DSA) , various radioisotope 
scanning, and nuclear magnetic resonance (NMR) imaging systems. 

The increased resolution afforded by such scanning systems allows direct identification 
of brain structures that could only be inferred from conventional roentgenological 
techniques. Stereotactic surgery, being primarily a procedure performed without the aid 
of direct visualization, is dependent on sophisticated imaging techniques for its 
accurate execution. Therefore, it necessarily follows that as computer and imaging 
technology improve, so do the possibilities of stereotactic surgery. The present 
invention is primarily concerned with the use of computer-graphics techniques and 
scanning techniques for generating various composite images to better aid the 
stereotactic surgeon in localizing structures, such as subcortical structures, lesions, 
or abnormalities. 

Non-computer systems have been developed in the art for stereotactic surgery. These 
systems use spatial coordinate determinations based upon the use of special plastic 
type grids and measurement devices. These systems are based upon hand plotting and 
calculation of coordinate positions. Some disadvantages to such systems' are that they 
are cumbersome, slow, relatively inaccurate, have very limited use, and are specific 
for only one type of scanning device and manufacturer. 

Software routines on hand-held calculators, e.g., HP41C, Sharp, and Epson HX20, have 
also been developed in the art for use with stereotactic surgery. Calculations are done 
by the use of a calculator instead of a grid and specific measuring instruments. 
Parameters for coordinate determination are entered into the calculator's functions by 
the use of similar grids. Some disadvantages of these systems are the same as the 
non-computer spatial coordinate systems described above. 

Some parent scanning devices, e.g., CT scanner, NMR scanner, etc., contain resident 
software systems. Rule grids are placed over the image in the scanner via a software 
graphics package of limited capabilities. Accuracy is limited, since measurements are 
done in "screen coordinates" and therefore the systems do not take into consideration 
various rotations of the patient's head or other body parts in the scanning device. 
These are purely systems for calculating coordinates and have no operative simulating 
capabilities. CT scanners and NMR scanners also have some image manipulation routines 
which consist of various means of ramping image grey scales for contrasting; however, 
they contain none of the other features of the present invention, such as brain 
anatomical mapping techniques, electrophysiological mapping, extensive image 
manipulation, image comparison, 3-D simulations, etc. By design, each of these systems 
is limited to use in a specific scanner and its use must be sanctioned by the scanner 
designer, such as shown in "The Role of Computed Tomographic and Digital Radiographic 
Techniques in Stereotactic Procedures for Electrode Implantation and Mapping, and 
Lesion Localization, " by T. M. Peters, et al . , Appl . Neurophysiol . , Vol. 46, pp. 
2 00-2 05 (1983) . Image sources from several different scanners cannot be compared. Too, 
all functions must be carried out in each specific scanner device which monopolizes the 
scanner's use. Another disadvantage is that improvements or modifications to these 
systems cannot be made as scanner manufacturers tend to resist or disallow the addition 
of non-proprietary software and/or hardware to their scanning systems since they are 
potentially at risk of incurring additional liability should the software and/or 
hardware not function as intended. 

Other prior art systems utilize IBM PC type computers, including clones, and other 
desktop type personal computer versions. In some of these versions, either a camera 
input interface is used to acquire scan images from an X-ray plate type hard copy into 
the computer's video display, or magnetic transport media is utilized, most notably 
magnetic computer tapes. One disadvantage of these systems is that these existing 
interface designs rely upon the digitized images acquired through a camera input. The 
camera is mounted over an X-ray type hard copy image of the scanner image section to 
obtain an image. These systems can be inherently inaccurate because they use various 
kinds of camera lens designs and different image aberrations result from different lens 
designs. These include warping and distortion of the image due to chromatic aberration, 
spherical aberration, coma, astigmatism, and various other aberrations which result 
from deviation of refracted or reflected light rays from a single focus, or their 
convergence to different foci, due to the spherical shape of the lens or mirror. It is 
difficult to adjust or compensate for these problems. Another disadvantage is that 
existing interface designs rely upon the transportation of digital data from the 
original scanning source via use of magnetic transport media, most commonly tapes or 
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work with older X-ray imaging technology and not with CT scans, NMR scans or PET or 
nsA The svstem drew map images, simulated probe trajectories and printed 
electrophysiological daL on the display terminal according to various parameters which 
were typed into certain program requests. 
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sJu^aart Thieme 1977) afforded an opportunity to expand the computer mapping 
capabflftiesto^nciude'rhombencephalir structures. Incorporating these maps into this 
system required the development of a coordinate system that would allow the 
Ill ltanZs display of diencephalic architectonics and the remaining brain stem and 
^erebenr AUowaSe was also^equired for variations in the angle of the 3 unction of 
the diencephaion with the lower brain stem. See Hardy, T- L... Koch, J.. 98 
■•Computer-assisted Stereotactic Surgery," Appl. Neurophysiol . Vol. 45, pp. 396 398, 
1982 which describes a software modification to the above-noted system described in 
III istl paper to allow a similar use of brain stem maps and describes the development 
of a brain stem coordinate system. Both coordinate systems could be moved independently 
of each other- the size of the brain maps, including the bram-stem length could be 
readily varied to match the patient -s anatomical dimensions as determined from contrast 
vesiculograms! ThVs was accomplished by developing a method of intersecting an upper 
TdiencephalicT coordinate system constructed about the third ventricular core with a 
lower coo~rainate system constructed about the fourth ventricle. For example the angle 
of intersection will be closer to 90 degrees in a patient with a brachiocephalic brain 
in llith brain stem angulation is perpendicularly oriented. Adjustments for difference 
in brain-stem sizes were achieved by a software subroutine that could be prompted to 
expand or contract the digitized maps. As with the above-noted system described in the 
1981 paper? this system wis not designed to work with newer waging technology After 
the development of high-resolution, color- graphics raster display raon ^ors that could 
interface with small computer systems it became possible to improve this system. The 
computer system was later modified so that it could use such a monitor to display CT 
imaaes as well as benefit from the addition of color graphics. The result was a 
portable svstem which could store, manipulate, and selectively display CT images in the 
operating room independent of the CT scanner. The previously digitized atlas maps also 
could be 9 supSrimposed on CT sections of the diencephaion. This method of graphic 
operative simulation served as a guide for using CT data in performing functional 
neurosurgery However, this system had to be programmed separately for each type of 
scanner-Computer Graphics with Computerized Tomography for Functional Neurosurgery, 
(T L Hardv J Koch, and A. Lassiter, Appl. Neurophysiol. Vol. 46, pp. 217-226 
M983)) describes a prototype system which could be used with CT imaging technology. 
TMS system could simulate plobe trajectories but could not determine coordinates. Such 
parameters had to be entered by typed entry into the computer's program. The system was 
capable of pseudocolor, but no other image manipulation routines were possible. CT 
imaae display was limited to four bits of image capabilities which gave a gray scale 
capability of 2 supT(16 levels). This system was fraught with difficulties and the 
hardware LsLn'was subsequently abandoned. Some of these difficulties were also due to 
tne cm ba-sed 9 o P erating system which was slow, cumbersome and ext remel y dif - 
use as a developmental platform. Hardy, T. L., Lassiter, A , and oc , . A Potable 
Computerized Tomographic Method for Tumor Biopsy", Acta. Neurochir. [Suppl.J , ^ien) , 
p 444? 19^3, describes the above-noted system as a laboratory model for simulating 
tumor biopsy. 

SUMMARY OF THE INVENTION 

In accordance with the present invention, there is provided a method and apparatus for 
aeneSing a vSeo presentation of images from a variety of separate scanner imaging 
fources The method of the invention comprises. the steps of acquiring a plurality of 
imaaes from a variety of separate scanner imaging sources, converting the acquired 
images into a selected format, storing the acquired and verted plurality of images , 
selectively recalling and displaying at least two of the stored plurality of i^ges, 
ani independently manipulating each of the selected images and comparing the selected 
imaaes The manipulating step preferably comprises shaping and sizing at least one of 
the selected images to conform to at least one other selected image in shape and size 
The method may further comprise superimposing at least one selected image upon at e 
one other selected image. The method may also comprise a step of comparing the selected 
tmages, contrasting characteristics of at least two selected images using at least one 
of the procedures of filtering, smoothing, sharpening, pseudocoloring and edge 
detection. The edge detection procedure can be, for example, a Laplacian, a Roberts a 
Sorbel or a Frei procedure. The image acquiring step can comprise obtaining at least 
one image directly from a scanner without the use of magnetic transport media. The 
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sequence of steps taken by a user can be recorded and archived. 

The method of the invention is preferably practiced in stereotactic surgery, and 
comprises generating a video presentation of brain images from a variety of separate 
brain scanner imaging sources to provide a representation of the anatomical and 
physiological configuration of the brain of a patient. The method comprises the steps 
of acquiring a plurality of brain-related images from a variety of separate brain 
scanner imaging sources, at least one such image being of the actual patient's brain, 
converting the acquired images into a selected format, storing the acquired and 
converted plurality of images, selectively recalling and displaying at least two of the 
stored plurality of images, and independently manipulating at least one of the selected 
images and comparing the manipulated image to at least one other of the selected 
images. The acquiring step can comprise obtaining images from a brain map atlas and the 
manipulation step comprises fitting at least one map from the atlas to at least one 
selected image of the patient ! s brain. The manipulating step can comprise shaping and 
sizing at least one of the selected images to conform to at least one other selected 
image in shape and size. The method can further comprise superimposing at least one 
selected image upon at least one other selected image. The step of comparing the 
selected images can comprise contrasting characteristics of at least two selected 
images using filtering, smoothing, sharpening, pseudocoloring, or edge detection. The 
edge detection procedure can be a Laplacian, Roberts, Sorbel, or Frei procedure. The 
method can additionally comprise the step of measuring the distance between two points 
within the actual patient's brain, the area of a selected portion of the brain, and the 
volume of a selected part of the brain. The method can comprise generating a simulated 
three-dimensional image of the actual patient's brain, the three-dimensional simulation 
image preferably being representative of an image such as a computerized axial 
tomography (CT) image, a nuclear magnetic resonance (NMR) image, a positron emission 
tomography ( PET) image, a digital subtraction angiography (DSA) image, isotope image, 
and an X-ray image. The three-dimensional simulation image comprises a composite of 
images from more than one source. The method can further comprise a step for 
determining optimum placement or dosage for an isodose implantation. An archiving step 
can be used to record and store the steps taken by a user. 

The invention additionally comprises an apparatus for generating a video presentation 
of images from a variety of separate scanner imaging sources. The apparatus comprises 
structure for acquiring a plurality of images from a variety of separate scanner 
imaging sources and for converting the acquired images into a selected format. Storage 
is provided for the acquired and converted plurality of images. Structure for 
selectively recalling and displaying at least two of the stored plurality of images, 
for independently manipulating each of the selected images and for comparing the 
selected images is also provided. The image manipulating structure preferably comprises 
structure for shaping and sizing at least one of the selected images to conform to at 
least one other selected image in shape and size. The apparatus can further comprise 
structure for superimposing at least one selected image upon at least one other 
selected image. The structure for comparing the selected images preferably comprises 
structure for contrasting characteristics of at least two selected images using 
filtering, smoothing, sharpening, pseudocoloring, or edge detection. The edge detection 
procedure can be a Laplacian, a Roberts, a Sorbel, or a Frei procedure. The image 
acquiring structure can comprise structure for obtaining at least one image directly 
from a scanner without the use of magnetic transport media. Storage or archiving means 
can be provided for storing the -user ! s procedural steps. 

The apparatus of the invention is preferably for use in stereotactic surgery and 
comprises an apparatus for generating a video presentation of brain images from a 
variety of separate brain scanner imaging sources to provide a representation of the 
anatomical and physiological configuration of the brain of a patient. The apparatus 
comprises structure for acquiring a plurality of brain-related images from a variety of 
separate brain scanner imaging sources, at least one such image being of the actual 
patient's brain, structure for converting the acquired images into a selected format, 
structure for storing the acquired and converted plurality of images, for selectively 
recalling and displaying at least two of the stored plurality of images, for 
independently manipulating at least one of the selected images and for comparing the 
manipulated image to at least one other of the selected images . The image acquiring 
structure preferably comprises structure for obtaining images from a brain map atlas 
and the image manipulation structure preferably comprises structure for fitting at 
least one map from the atlas to at least one selected image of the patient's brain. The 
manipulating structure preferably comprises structure for shaping and sizing at least . 
one of the selected images to conform to at least one other selected image in shape and 
size. The apparatus can further comprise structure for superimposing at least one 
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detection. The edg. detection P™«*£ h .^ 0 *! r le e £"cture £0° emulating . br.in 
presentation. The apparatus can *Y rt *« ~™P" ei ec ^des within a video presentation, 
physiological response point. «* «~ 7 ,p" tu . %,„ further comprise 

presentation. The image acquiring Bt ^ ctu " . "pEl'scanner, an X-ray scanner, 

Suture t^l ^in g ra opti m urp?ace::nt C o; dosagf for an isodose implantation. A 
reccing or archiving device for storing user procedure can be provided. 
One object of the present invention is to provide a user with a video presentation of 
an image from one of a variety of imaging sources. 

mother- obiect of the present invention is to convert images from a variety of 
incom^atibirsources into a common format for storage, display and comparison. 

55^ doping oL-r ^JX^^**^ 2^"* " 
— rS^^^^ 

Another advantage of the present invention is that a variety of images from different 
sources can be readily compared. 

the value of such images to the user. 

a variety of sources during a procedure. 

each was enhanced, manipulated and otherwise used. 

b£ means of the instrumentalities and combinations particularly pointed out in the 
appended claims. 
DRAWING DESCRIPTION: 

BRIEF DESCRIPTION OF THE DRAWINGS 
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The accompanying drawings, which are incorporated into and form a part of the 
specification, illustrate a preferred embodiment of the present invention and, together 
with the description, serve to explain the principles of the invention. In the 
drawings : 

FIG. 1 schematically illustrates a preferred method of the invention; 

FIG. 2 illustrates a preferred hardware block diagram in accordance with the invention; 

FIG. 3 is a perspective view of the preferred apparatus of the invention; and 

FIGS. 4-12 show various brain images depictable on the monitor of the FIG. 3 apparatus. 

DETAILED DESCRIPTION: v 

l' 

DETAILED DISCUSSION OF A PREFERRED EMBODIMENT OF THE INVENTION 

The computer assisted stereotactic surgery apparatus and method of the present 
invention comprise hardware components and associated software, providing to a 
stereotactic surgeon the ability to acquire digital images from many different scanner 
imaging sources, to store these images in a computer system independent of the original 
scanning device, and to display and manipulate the stored images, as desired. The 
invention is useful for performing functional and morphological > stereotactical 
procedures, including but not limited to tumor biopsies, thalamotomies, such as for the 
treatment of dyskinesias, pain syndromes, seizure disorders, blood vessel 
abnormalities, and the diagnosis and treatment of a variety of other structural or 
functional abnormalities. \ 

In the preferred embodiment, the images are obtained and converted to a common standard 
format, such as an RS-170 format, for comparison, independent manipulation and display 
with other graphics capability at any selected time, such as during a stereotactic 
operative procedure. The surgeon, therefore, has in one system, means of acquiring, 
comparing and manipulating images from many scanning sources, and can utilize this data 
to plan and perform stereotactic surgical procedures in a cost effective manner. A 
surgeon is able to interface directly with the system in a sterile fashion during an 
operative procedure. This latter capability is afforded in the preferred embodiment 
preferably by an infrared beam touch screen interface through software icons for system 
manipulation. In practicing the invention, the surgeon can use the various images and 
data independent of the original scanning devices. 

The preferred embodiment of the invention can be used by a surgeon with a limited or 
rudimentary understanding of computer systems. Indeed, it is not required that a 
surgeon have any knowledge of computer systems in order to operate the preferred 
embodiment. This frees a surgeon of this additional and extensive responsibility. 

The method of the invention is illustrated in FIG. 1. The method of the invention 
comprises scanning 10 the patient's head or other body part(s); image acquisition 12 of 
the scans obtained; image format conversion 14 for all images so that they are 
converted into the same format; image storage 16; multiple image recall and display 18 
of the images; and image manipulation and comparison 20. 

The preferred steps for practicing the method of the present invention are as follows: 

a) Provide the apparatus in accordance with the invention in a scanner suite and 
connect the apparatus to a desired scanner, such as a CT, NMR, DSA, or isotope output 
scanner; 

b) mount a stereotactic frame on the patient; 

c) place the patient's head or other body part(s) to be analyzed in the scanner; 

d) acquire the images desired using the scanner; 

e) save or store the acquired images in the apparatus; 

f) take the apparatus and the patient into the operating suite; 

g) 'use the image manipulation routines in the apparatus to calibrate and preferably 



11 of 24 



4/30/02 2:55 PM 



wysiwyg^/http^ 

Record Display Form 

further enhance the desired image areas; 

h) ,„ y of th. .pp.r.tue £,in ».p. S;^^ 1 L 1 JLcr.^<inr^.r g .t in 
S"l«ion for morphologic! procedures; 

„ u ee t „= » --IXlor^'i^t^/SnSS-rSUoLcUc 

point, or placement and trajectory f 
or morphological procedures; and 

p „p«, . tap. archive of -tb. encir. operative procedure, if -tor,,, of d... ». 
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desired. 



aesireu. . . 

pjQ 2 iuu.tr.t- a preferred hardware ^loc, diagram in accordance with^he^n^^ - 
ScannersSo. such as CT, magnetic r.-jnjnc^jW data '. This .canning data, 

isotope scanners, are provided to obtain ^ari convertible to a standard 

which is in various, typically non-standard formats ideQ si al processor 

format in accordance with the «^Jf ^farfnot limited to, RS-170, CCIR 

32 Standard formats which can be used in ^ u « e ' ° be prov ided to the computer 

Ilk I RS-343, RS-422, and NTSC . The scanning data may De pr connec tor. The 

Seo signafprocessor 32 from the scanners 30 by Jj^Sj; a relay control 3S to a 
converted scanning data is made mailable, ^^20/03 0 VME BUS based CPU with a clock 
central processing unit (CPU 34 su h as a 68 / fioppy disc storage, 

speed up to 30 MHz. P refe ^ ablv . C ° m P^^C?ion frame grabber with a high speed graphics 
streamer tape storage, and a h^'^i^*^*™ control 35 sets the scanner converter 
and video image processor. The P^^r^^^f^^^r image data from its various 
input/output parameters ac ^\ s f by °^ System in accordance with the invention, 
formats to a standard format, f °* u "° 2 a tically senses signal input and sets 
Alternatively, a scanner converter *^omatical± Y * use of user selectable 

parameters for conversion to % f ^^J 0 ™** ' Resolution infrared touch screen and a 
switches. A user input device 38 such as a hig ^ selectively manipu late and 

monitor 36, both connected to the CPU 34 *™ b £ high-resolution color graphics 

display the image data. The monitor 36 is P^ er / c 0 site video signals. The 
monitor with video inputs fo / ^ed^reen, blue a inte rlaced or non-interlaced mode 
^erofimage^^^ STSVSvS- using a hard copy device 40, such as a 

high resolution color videocopy unit. 
Software, provided in . the ^-^^ 

bi-divisional foundation which c°*P"ses a divi user specific functions. The division 
and manipulation and a division f ° r S^nioulation includes modular software ^ 
for image acquisition, enhancement a * d manl P^f Chiving; 2) pixel analysis for an 
subroutines for: 1) image capture forage and archivi P functions; 4) 

entire image or user-def ar f^ °_ ^notions for smoothing, sharpening, and 
contrasting and filtering i ma 9* s ^ imaqe editing; and 7) various edge detection 

pseudocoloring; 5) image comparisons image ed g, _ The graphics and 

routines, including Laplacian Roberts Sorbel subroutines that control: 1 the 
user- specific functions comprise software m °* U £ ar . tem atlas maps (preferably at 
manlpuLtion and swapping of d^l^sajittaf horizontal sections; 2) the graphic 
least 117 of such maps) in frontal, sagittal , ana electro des; 3) alphanumeric 

simulation and the manipulation of ^"^^SoSng! S selectively displaying the 
functions for demographic data and ^"J^^^gi. on an electrophysiological map 
position of various ^^"^"faf leLt 2500 electrophysiological response points; 
system which preferably cora P r ^^^ n i n ^ 8ter eotactic coordinates and other 
user^cifif calcuiatiLsranfrvoSetric determinations and two- or 
three-dimensional simulations. 

ninchratPd in FIG . 3. The apparatus 
The preferred apparatus of the "^^.^"JS "fusing the CPU 34, the monitor 36, 
comprises an outer case 50 and an inner case 52 t ^ 62 A foam laye r 54 for 

the hard copy unit 40, and the ^^"^f^fw transported is disposed between the 
absorbing shock so that the ^P 3 "^ 8 ""^ controls 56 for adjusting, for instance, 
outer case 50 and the inner case 52. Monitor £? n "° n * 36 prefe rably has disposed 
brightness, gain, and degauss, are Prided The monitor p ator can i de 

thereon a touch screen user interface 58 so that tne s g . storage modules 60 

input merely by touching the monitor screen with . styl^ * 62 converts 

fhrscLnin/rat^nto ^standard"* ormaf f orYmage enhancement and manipulation. A 
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storage drawer 64 for storing tools, or the like, may be provided in the apparatus for 
ease of access by the operator. Handles 68, along with wheels or casters 66 on the 
bottom of the apparatus enable the apparatus to be easily moved, for instance from the 
scanning room to the operating room. Software usable with the apparatus hardware to 
practice and to carry out the desiderata of the invention is disclosed herewith. 

The program listings in the microfiche appendix are directed to the programmed 
interface to computer video processor, an electrophysiological maps program, an 
anatomical maps program, a probe placement program, a stereotactic frame calibration 
program, an image overlay program, an icon layout program and a touchscreen control 
program. These programs are unique to the invention. Other programs used in practicing 
the invention, such as image manipulation programs, are either commercially available 
or within the skill of practitioners in the programming arts. 

Although a brain anatomical map system was present in a rudimentary form, as published 
in the paper of Hardy, Koch, Lassiter (1983), it has subsequently been extensively 
improved for use with the invention. 

In practicing the invention, a surgeon can acquire images by a direct coaxial link to 
various scanners and thereby avoid the cumbersome use of magnetic transport media. 

At least two images should be obtained when the apparatus and method of the invention 
are used, one image at the level of the intended analysis and another image at the 
level of the entry point obtaining additional images enhances the accuracy and 
efficacy. These additional images are necessary for volumetric determinations. 
Volumetric determinations are helpful, for example, if the tumor is to be treated using 
implantation of radioactive isotopes. Preferably the images are stored and not lost 
when the apparatus is turned off, so that the apparatus can be easily transported from, 
for example, the scanner suite to the operating suite. This can be accomplished, in 
accordance with the invention, by storing the images on a hard disk. 

For thalamotomies, it is useful to obtain images in approximately 1 . 5 mm increments in 
an area of between approximately 20 to 30 mm above the posterior clinoid process to 
include both the anterior and posterior commissures. Once images are acquired through 
the commissures, the thickness of the image sections can be increased both above and 
below the thalamic zone. This method reduces scanning time and decreases the radiation 
dose (when CT scanning is utilized) to the patient. The apparatus and method of the 
invention can be used to obtain a horizontal reformatted image along the 
intercommissure plane showing both the anterior and posterior commissures in the same 
image section. Most stereotactic surgeons regard the nucleus ventralis intermedius 
(V.i.m.) of the thalamus as the anatomical target site of choice for lesions in the 
treatment of dyskinesias. Other stereotactic surgeons prefer to extend the lesion 
slightly inferiorly into the dorsal subthalamic H-fields of Forel . The general target 
is small, and measures approximately 6 mm. times. 4 mm. times. 4 mm. The more inferior a 
lesion is made, the smaller it has to be to treat dyskinetic movements. Conversely, the 
higher a thalamic lesion is made, the larger it has to be to get equal results. The 
target area is bounded by important anatomical functional zones; that is, the motor 
fibers in the internal capsule laterally, the sensory nucleus posteriorly, the 
mammillothalamic bundle anteriormedially, and the subthalamic nucleus inferiorly. The 
invention allows for accurate lesion localization. 

Fiducial markings should be present on all images so that calibration and probe 
positioning determinations can be achieved. Preferably, the surgeon can select the 
individual images to be calibrated. Frame calibration techniques, such as tacking 
fiducial marks on the images, are particularly useful in accordance with the invention. 

Various electrodes or probes, common to the art, are useful in accordance with the 
invention. Two methods are useful in the preferred embodiment for defining the 
trajectory of the probe. In the first method, the entry point of the probe is 
designated by tacking it on the image. A target point on another scanner image slice is 
selected and the coordinates of the two points can then be used to determine a probe's 
spatial trajectory. In the second method, the trajectory of the probe can be defined by 
entering the probe's angles according to the stereotactic frame design, such as the 
anterior-posterior and lateral angles using a Leksell frame, or the angles of azimuth 
and declination (alpha, beta, delta, and gamma angles) using a Brown-Roberts-Wells (a 
registered trademark) frame. A target point on one image slice is chosen. Then the 
angles of the probe's spatial intersection with that point are entered. This defines 
the probe's spatial trajectory. The image acquisition and storage preferably takes 
approximately 15 minutes for approximately 20 images. 
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further illustrated by the following non- limiting example. 



A preferred appar.tu. in "^^IJ/^l^^ 1 ^"-^'^.^ 

are E p^t£ r .^n» L lockel to pr.ven? unwanted .cc.es to the apparatu. The 

* ie ^ / ^ _ j _ j • fM^pv ^acA bv an inner case or 521/z mgn, 

ILZhila foam This configuration allows for easy transport of the apparatus with 
absorbing foam. This con "f u ^ is airline approved and therefore can also act as a 
King^oSinS^or Stratus. Individual apparatus components can be fitted 
tnllthl rackmount inner case*. Such cases can be custom made from commerce! 
manufacturers . 

i fl a nq ii raclcmount hiah resolution (800 . times . 1024 or 

™iLTcSor*£p»™. ""'itot with S.rd vfdeo input, to red. or.en. blue, and 

incerxdteu 00.343 video standard. Video display can be set at a 1 : 1 or 3.4 

video standard or RS 343 video scan v 1 ^ ^ shor( . istence 

tne user The brightness and gain can be used to vary the apparent intensity of an 

tne user, me uij-ynu „„^v. fV,^ it nan be easilv viewed in various ambient 

image appearing on the -£een *uch th at - S * s y Jp 0rtan t since the system can 

the system's computer. The monitor's RGB and composite video inputs can be looped 
ohms termination. Such monitors are standard and commercially available. 

-i£rw^^^ 

aTarr.y of infrared diode, '"^-"".""f .'^Thl."^^™ .'droned .0 
in the commands which can be executed by that icon. 

Th* color video thermal hard copy unit is commercially available and is capable of 

E&«S "de^ini/or^e. • 
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illustrated in FIGS. 4 12. FIG. 4-12 are color images of the brain obtained by the 
apparatus of the invention through a computer display. FIG. 4 is a saggital NMR image 
with brain map overlay and simulated probe positioning for target. FIG. 5 is a 
magnified display showing brain maps and electrophysiological maps. FIG. 6 is an image 
from a computer display showing a CT image with fudicial markings about the periphery . 
FIG. 7 is an image similar to FIG. 6, which shows deep seated brain tumor with 
simulated probe trajectory noted. FIG. 8 is a CT image with the Leksell stereotactic 
frame and associated fudicial markings, where no probe simulation is shown. FIG. 9 is 
an NMR image of the brain. FIG. 10 is another NMR image of the brain. FIG. 11 is a PET 
image of the brain. FIG. 12 is a DSA image of the brain. The target coordinates of 
FIGS. 4-12 are shown for each on the image of the computer display. 

The unit's color selection range, intensity, brightness, and contrast can be controlled 
by the user through the use of selected control knobs housed in the front of the unit. 
This unit is used to make selected hard copy archives of images displayed on the CRT 
screen. An additional advantage of this particular unit is that it is small, compact, 
easily serviced and used by the end user. Paper and film can be easily placed in the 
system by the use of cartridge loaders and is therefore easy to prepare by the user. 

The CPU comprises the various hardware components and software components for the 
central functioning of the apparatus. All hardware components are commercially 
available and have been integrated in such a fashion to achieve the desired features 
and functions of the invention. Various components of the computer processing unit are 
as follows: 

Component 1: Central Processing Unit (CPU). The central processing unit consists of a 
VME bus based computer as is commonly manufactured by Motorola Corporation and other 
corporations in the United States and is a European Common Market standard where it is 
called the EuroCard. The present system runs on a 68020 processor at 16.67 mHz . The 
system has 8 megabytes of dynamic random access memory (DRAM) . This particular system 
configuration, that is, the VME bus, was selected for ease of design, future upgrades 
and service. It must be noted that the software routines of the invention can be used 
on other computers; for example, IBM registered trademark PC, AT, XT, or P2 , or such 
similar systems, the DEC VAX registered trademark, SUN VME systems or other such 
similar systems. The software can also run on considerably faster CPUs. The central 
processing unit is configured such that the system automatically boots itself up upon 
power-on. It has at least one RS-232 port through which communication is directed to 
and from the infrared touchscreen. 

Component 2: Archiving and Storage Modules. The archiving and storage modules consist 
of a 5l/4"l mb floppy disk drive; at least an 86 mb Winchester hard disk drive; and at 
least a 60 mb streaming tape cartridge tape drive. These components are used to house 
the system's software operating system and programs (Winchester hard disk) and to save 
patient and image data. The system 1 s operating system is a standard UNIX base software 
system which is either an AT&T or Berkeley UNIX standard which is commercially 
available . 

Component 3: Image Acquisition Interface. The image acquisition interface consists of 
several components which are: 

a) VME bus base framegrabber video display board with a minimal configuration of 12/16 
bit graphics overlay display capability. This component is commercially available and 
is capable of converting a standard (RS-170) or non-standard video signal to a digital 
format which is stored in its memory. Such images are stored as an 8 bit deep image 
with gray scales of 256 levels. Such stored images can then be displayed through either 
a red, green, blue (RGB) video output or a composite video output in either interlaced 
or non-interlaced mode to the high resolution color graphics monitor. The board has at 
least four to eight additional bit planes available for graphics overlay. This 
configuration allows the simultaneous display of multiple images (combination of 
scanned images or graphic images) on the same monitor for composite viewing. In 
addition, such images can be independently manipulated as subsets without interference 
to other images. This configuration allows the very rapid and non-destructive 
manipulation of multiple images in a common display. Image output is as a standard 
RS-170 video format. This board will accept inputs from a standard videoscan converter, 
camera interface, or video tape input or directly from the system's CPU. 

b) VME bus based Ethernet board. This board is a commercially available communications 
data acquisition board which can transfer digitized non-standard video input data in a 
standard format (ARC-NEMA) from a common communication network between various scanning 
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devices and store them as digital data in memory. Images acquired in this fashion and 
stored in the computer system's memory can then be transferred to the memory of the 
framegrabber video display board as noted in a) above. Images from non-standard video 
can be digitized in this fashion for eventual display in a standard format; for 
example, CT scans, NMR scans, PET scans, isotope scans, and others. Video inputs 
through this board do not allow real time viewing but are capable of giving a slightly 
clearer image. In addition, this configuration allows the invention to acquire and 
utilize images from a common network which may be present in some hospital settings. 
This represents an alternative source from image acquisition and conversion from a 
common format as noted in c) below. 

c) Computer video processor (CVP) . This component is commercially available and is 
capable, apart from the Ethernet board noted above , of converting all non-standard 
video signals into a common RS-170 video format. This component is configured by the 
programming of a special chip in its own separate central processor to acquire a wide 
variety of non-standard video signals within a certain designated range. As used in the 
invention, the range is set so that it can convert signals for CT scans, NMR scans, PET 
scans, DSA, etc., into a common format. The parameters which can be set preferably 
include at least all of the following signal parameters: 



1. horizontal front porch 

2. horizontal back porch 

3. vertical front porch 

4. vertical back porch 

5. horizontal blanking time 

6 . vertical blanking time 

7. pixel clock rate 

8. horizontal frequency 

9. vertical frequency 

10. horizontal sync width 

11. vertical sync width 

12. vertical refresh rate 



13 . composite video voltage level 

14. non-composite video voltage level 



15. line rate 



The input to this unit is via a standard 75 ohm BNC connector from various scanner 
imaging sources. Such scanning imaging sources are then processed by the central 
processor unit portion of the computer video processor and converted to RS-170 standard 
signal output. The signal output as RS-170 format is inputted into the framegrabber 
video display board for further processing and output. Images inputs through this part 
of the system allow continuous real time viewing. The clarity of some images may be 
slightly reduced due to signal averaging. The CVP has external switching capabilities 
which can set the specific parameters as noted in the above list . As commercially 
available these external switches are 16 in number and consist of: 

1. Preset field blanking #1 



2. Preset field blanking #2 

3 . Preset input divider #1 

4 . Preset input divider #2 

5. 32 kHz (nomial mode) 
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6. Custom preset parameters (special EPROM chip interface. 

7 . Tester mode 

8 . Freeze frame 

9. Swap fields 1 and 2 

10. Decrement field blanking 

11. Narrow input clock 

12 . Widen input clock 

13 . Move image left 

14 . Move image right 

15. Narrow output clock 

16 . Widen output clock 

A programmable switching interface which consists of a VME bus based digital relay 
board with 16-32 independent switches can be programmed for activation through the 
computer. The interface between the relay board and the switch input is via a specially 
wired input /output box between these two levels of communication. Alternatively, a 
scanner converter automatically senses signal input and sets parameters for conversion 
to a standard format, without the use of user selectable switches. 

Data entering into the system is from various scanning modalities via a standard 75 ohm 
video cable with standard BNC connectors. This cable carries a composite video output 
from any of the respective scanning devices and enters directly into the VME bus based 
framegrabber video display board (imaging board) if it is an RS-170 (CCIR or NTSC) 
video standard, for example, a standard RS-170 video camera input, recording device or 
some scanners which output an RS-170 standard. If the output from the scanner is a 
non-standard video output, e.g., CT scan, NMR scan, PET scan, DSA scan, and others, 
then the input is into the computer video processor unit. At the level of the computer 
video processor unit, the non-standard video signal is analyzed and converted by the 
system's signal processor and converted to a standard RS-170 video output which then is 
output to the VME bus based framegrabber display board. In addition, at the level of 
the computer video processor specific switchable parameters for analysis and conversion 
of the incoming non-standard composite video scanner output are set via a relay control 
board which is selectable through a programmed interface in the apparatus computer. 
Software in the system allows such parameters to be set so that the user can indicate 
which scanner images are being acquired. In some cases the RS-170 video standard output 
or other non-standard video outputs, e.g., RS-343, RS-344, from various scanning 
devices can either be in the form of a composite video output signal or as a TTL format 
with separate horizontal and vertical sync outputs and a pixel clock pulsed output. The 
system is capable of acquiring images in either manner. 

An alternate source of image acquisition may be obtained from a common Ethernet 
communications network between various scanners. Data from this communications network 
may be available as a common standard, ARC-NEMA, which the system through its Ethernet 
board may acquire images for its use. In such cases, the computer video processor would 
be by-passed and images acquired through the Ethernet board communications network is 
interfaced directly with the video display board. 

At the level of the VME bus based framegrabber video display board the RS-17 0 video 
standard input (or input via the Ethernet) is digitized in frame memory in an 
512 .times . 512 .times . 8-bits deep format. With this configuration images can be stored 
with up to 256 gray scale pixel levels. Images digitized into the framegrabber board 
can then either be stored in the system's dynamic memory or in the system's memory 
storage units (Winchester disk, floppy disk, or streaming tape) . Images stored in such 
fashion are archived by the software system and can be recalled for rapid display on a 
19- inch high resolution color graphics video display monitor. The output to the monitor 
is via a standard RGB color video output with the sync pulse on the green output. 

The CPU controls all data inflow and outflow through its special software programs. The 
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image display board has 4 to 8 bits of image overlay capability which the system 
utilizes through its software routines to make composite images, for example, graphic 
overlays, brain maps, probe, electro-physiological maps, measurements, alpha- numeric 
data and various calculated parameters. One set of software routines listings, used in 
accordance with the invention, are appended hereto in the microfiche appendix and are 
hereby incorporated by reference into the specification. These routines include the 
following: Programmed interfaces to computer video processor; electro-physiological 
maps program; anatomical map program; probe placement program; stereotactic frame 
calibration program; image overlay program; icon layout program; and touchscreen 
control program. The image manipulation programs are commercially available. 

The CPU's functions are directed by the user through the use of an infra-red 
touchscreen which has an input via a standard RS-232 8 -bit input-output serial 
interface into the CPU. The user directs the functions of the unit by pointing at 
various icons (picturegrams) which are displayed on the monitor's cathode ray tube. The 
act of pointing at an icon displayed on the CRT screen within the confines of the 
infra-red touchscreen matrix directs the CPU to institute a predetermined series of 
programmed functions specific to the user's desires and within the parameters of the 
then existing software design. 

Through a standard loop-through off of the high resolution color graphics monitor, a 
standard video thermal hard copy unit is attached for the production of hard copy 
archives. The user can activate this unit by selecting certain switchable parameters on 
the front of the unit. 

The invention has been described in detail with particular reference to a preferred 
embodiment thereof, but it will be understood that variations and modifications can be 
effected within the scope of the invention. 

CLAIMS : 



I claim: 

1. An apparatus for generating a presentation of images from a variety of imaging 
sources, the apparatus comprising: 

means for acquiring a plurality of images from a plurality of separate imaging sources; 



means for converting said plurality of images into a selected format; 
means for storing said plurality of images; 

means for selectively recalling and displaying at least two images of said plurality of 
images upon a single display device; 

means for manipulating at least one of said at least two images independently of the 
other image; 

means for comparing said at least two images; 

means for determining stereotactic coordinates and performing volumetric determinations 
from said at least two images; and 

means for determining distances and areas from said at least two images. 

2. The apparatus of claim 1 wherein said means for acquiring a plurality of images 
comprises means for acquiring body part-related images with at least one of said images 
being of an actual patient's body part. 

3. The apparatus of claim 2 wherein said means for acquiring body part-related images 
comprises means for acquiring brain- related images with at least one of said images 
being of the actual patient's brain. 

4. The apparatus of claim 3 wherein said means for acquiring said brain-related image 
further comprises means for obtaining at least one image from a brain map atlas. 

5. The apparatus of claim 4 wherein said brain map atlas comprises a composite of brain 
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maps . 

6. The apparatus of claim 4 wherein said means for manipulating at least one of said at 
least two images comprises means for manipulating said image from the brain map atlas 
to conform to said image of the patient's brain. 

7. The apparatus of claim 3 further comprising means for displaying a representation of 
the patient's brain selected from the group consisting of anatomical configurations, 
physiological configurations, electrophysiological configurations, and combinations 
thereof . 

8. The apparatus of claim 3 wherein said means for determining distances and areas 
comprises means for obtaining within the actual patient's brain, the distance between 
two points within the patient's brain and the area of a selected portion of the 
patient 1 s brain . 

9. The apparatus of claim 3 wherein said means for determining distances and areas 
comprises means for obtaining within the actual patient's brain, the volume of a 
selected part of the patient's brain. 

10. The invention of claim 9 further comprising means for simulating a surgical device 
and determining trajectory of said device comprising: 

means for locating the target point on an image of the patient's brain; 

means for intersecting said target point by deriving the anterior-posterior and lateral 
angles of the device, wherein said angles are derived by using a stereotactic device. 

11. The apparatus of claim 10 wherein said means for intersecting said target point 
comprises means for deriving azimuth and declination angles of said stereotactic 
device . 

12. The apparatus of claim 11 wherein said azimuth and declination angles are derived 
using said stereotactic device. 

13. The apparatus of claim 3 wherein said means for manipulating said images further 
comprises means for manipulating and swapping brain anatomical images in frontal, 
sagittal and horizontal sections. 

14 . The apparatus of claim 1 wherein said image acquiring means comprises means for 
obtaining at least one said image directly from at least one imaging source without the 
use of magnetic transport media. 

15. The apparatus of claim 1 wherein said image acquiring means comprising media means 
for obtaining at least one said image by the use of magnetic transport. 

16 . The apparatus of claim 1 wherein said image acquiring means comprises a direct 
coaxial link with at least one imaging source. 

17. The apparatus of claim 1 wherein at least one imaging source comprises a scanner 
source selected from the group consisting of computerized axial tomography (CT) 
scanners, nuclear magnetic resonance (NMR) scanners, positron emission tomography (PET) 
scanners, isotope scanners, digital substraction angiography (DSA) scanners, and X-ray 
scanners . 

18. The apparatus of claim 1 wherein at least one imaging source comprises a video 
source . 

19. The apparatus of claim 1 wherein said recalling and displaying means comprises 
means for selectively recalling and displaying an image comprising references for 
stereotactic localization. 

20. The apparatus of claim 1 further comprising user means to input data within a 
sterile environment. 

21. The apparatus of claim 1 wherein said means for acquiring a plurality of images 
comprises means for acquiring a predetermined number of image types . 

22. The apparatus of claim 21 wherein said means for acquiring a predetermined number 
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of image types comprises means for acquiring different image types from a single 
imaging source. 

23. The apparatus of claim 1 further comprising means for manipulating both of said at 
least two images independently of each other. 

24. The apparatus of claim 1 wherein said means for manipulating at least one of said 
at least two images comprises means for shaping and sizing at least one of said at 
least two images. 

25. The apparatus of claim 24 wherein said means for manipulating comprises means for 
shaping and sizing at least one of said images to conform to at least one other said 
image in shape and size. 

26. The apparatus of claim 1 further comprising means for superimposing at least one of 
said at least two images upon the other of said at least 'two images. 

27. The apparatus of claim 1 wherein said manipulating means comprises at least one 
means selected from the group consisting of filtering, smoothing, sharpening, 
pseudocoloring, and edge detection means. 

28. The apparatus of claim 27 wherein said edge detection means comprises means for 
executing at least one routine selected from the group consisting of Laplacian, 
Roberts, Sorbel, and Frei routines. 

29. The apparatus of claim 1 further comprising means for recording and archiving the 
procedural use of said apparatus . 

30. The apparatus of claim 1 further comprising means for providing stereotactic 
coordinates for a user selected point on the presentation. 

31. The apparatus of claim 1 further comprising means for simulating at least one 
image . 

32. The apparatus of claim 31 wherein said simulating means comprises means for 
simulating at least one acquired image. 

33. The apparatus of claim 31 wherein said means for simulating is selected from the 
group consisting of simulating at least one two-dimensional graphic image, and 
simulating at least one three-dimensional graphic image. 

34. The apparatus of claim 31 wherein said simulated image comprises a composite of 
images . 

35. The apparatus of claim 34 wherein said simulated image comprises a composite of 
manipulated images. 

36. The apparatus of claim 35 wherein said simulating means comprises means for 
independently manipulating components of said composite of images as subsets. 

37. The apparatus of claim 36 wherein said simulating means comprises means for 
manipulating said subsets independently of other said images. 

38. The apparatus of claim 1 wherein said displayed images comprise a composite of 
images . 

39. The apparatus of claim 38 wherein said composite of images comprises at least one 
said acquired image and one simulated image. 

40. The apparatus of claim 39 wherein said recalling and displaying means comprises 
means for independently manipulating said acquired image and said simulated image. 

41. The apparatus of claim 1 further comprising an infrared grid disposed across said 
image displaying device. 

42. The apparatus of claim 1 further comprising means for simulating at least one 
device selected from the group consisting of stereotactic devices, probes, electrodes, 
implantation devices, and needles. 
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43. The apparatus of claim 42 further comprising means for simulating manipulation of 
said device. 

44. The apparatus of claim 42 wherein said device comprises an isodose implantation 
device and said apparatus further comprises means for determining optimum placement for 
isodose implantation, means for determining optimum dosage for isodose implantation, 
and means for simulating an isodose implant. 

45. The apparatus of claim 1 wherein said means for determining stereotactic 
coordinates further comprises means for determining stereotactic coordinates of at 
least one entry point, target point, placement point and trajectory of a device. 

46. The apparatus of claim 1 further comprising means for pixel analysis. 

47. The apparatus of claim 46 wherein said means for- pixel analysis comprises means for 
pixel analysis of an image selected from the group consisting of an entire image, a 
user-defined area of an image and a combination thereof. 

48. The apparatus of claim 47 further comprising means for selecting image data on said 
images on the basis of said pixel analysis. 

49. The apparatus of claim 48 further comprising means for generating composite images 
from said image data so as to provide at least one representation selected from the 
group consisting of anatomical configurations, physiological configurations, 
electrophysiological configurations, and combinations thereof, of a patient's brain. 

50. The apparatus of claim 1 wherein said means for recalling and displaying comprises 
video presentation means . 

51. The apparatus of claim 1 wherein said means for recalling and displaying comprises 
means for displaying images independent of the imaging sources. 

52 . The apparatus of claim 1 further comprising means for coding and storing 
representations of physiological response points and means for selectively recalling 
and displaying any of said response points within the presentation. 

53. A method for generating a presentation of images from a variety of imaging sources, 
the method comprising the steps of: 

acquiring a plurality of images from a plurality of separate imaging sources; 
converting the plurality of images into a selected format; 
storing the plurality of images; 

selectively recalling and displaying at least two images of the plurality of images 
upon a single display device; 

manipulating at least, one of the at least two images independently of the other image; 
comparing the at least two images; 

determining stereotactic coordinates and performing volumetric determinations from the 
at least two images; and determining distances and areas from the at least two images. 

54. The method of claim 53 wherein the step of acquiring a plurality of images 
comprises acquiring body part- related images with at least one image being of the 
actual patient's body part. 

55. The method of claim 54 wherein the step of acquiring body part-related images 
comprises acquiring brain-related images with at least one of the images being of the 
actual patient's brain. 

56. The invention of claim 55 wherein the step of acquiring the brain-related image 
further comprises obtaining at least one image from a brain map atlas. 

57. The method of claim 56 wherein the step of obtaining at least one image from a 
brain map atlas comprises the step of obtaining at least one image from a composite of 
brain maps. 
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58. The method of claim 56 wherein the step of manipulating at least one of the at 
least two images comprises manipulating the image from the brain map atlas to conform 
to the image of the patient's brain. 

59. The method of claim 55 further comprising displaying a representation of the 
patient's brain selected from the group consisting of anatomical configurations, 
physiological configurations, electrophysiological configurations, and combinations 
thereof . 

60. The method of claim 55 wherein the step of determining distances and areas 
comprises obtaining within the actual patient's brain, the distance between two points 

of a selected portion of the patient's brain. 

61. The method of claim 55 wherein the step of determining distances and areas 
comprises obtaining within the actual patient's brain, the volume of a selected part of 
the patient's brain. 

62. The method of claim 61 further comprising simulating a surgical device and 
determining trajectory of the device comprising: 

locating the target point on an image of the patient's brain; and 

intersecting the target point by deriving the anterior-posterior and lateral angles of 
the device, wherein the angles are derived using a stereotactic device. 

63 . The method of claim 62 wherein intersecting the target point comprises deriving 
azimuth and declination angles of said stereotactic device. 

64. The method of claim 63 wherein the azimuth and declination angles are derived using 
said stereotactic device. 

65. The method of claim 55 wherein the step of manipulating the images further 
comprises manipulating and swapping brain anatomical images in frontal, sagittal and 
horizontal sections. 

66. The method of claim 55 further comprising actually executing functional and 
morphological stereotactical procedures for brain targets selected from the group 
consisting of dyskinesias, pain syndromes, tumors, lesions, aneurysms, seizure 
disorders, functional anatomical sites, physiological response points, blood vessels), 
abnormalities and neuropathological conditions. 

i 

67. The method of claim 52 wherein the step of acquiring images comprises obtaining at 
least one of the images directly from at least one imaging source without the use of 
magnetic transport media. 

68. The method of claim 52 wherein the step of acquiring images comprises obtaining at 
least one of the images by the use of magnetic transport media. 

69. The method of claim 52 wherein the step of acquiring images comprises providing ja 
direct coaxial link with at least one imaging source. 

70. The method of claim 53 wherein the step of acquiring images comprises the step of 
acquiring images from at least one imaging source comprising a scanner source selected 
from the group consisting of computerized axial tomography (CT) scanners, nuclear 
magnetic resonance (NMR) scanners, positron emission tomography (PET) scanners, isotope 
scanners, digital substraction angiography (DSA) scanners, and X-ray scanners. 

71. The method of claim 53 wherein the step of acquiring images from at least one , 
imaging source comprises the step of acquiring images from a video source. 

72. The method of claim 53 wherein the step of acquiring images further comprises the 
step of providing references for stereotactic localization. 

73 . The method of claim 53 further comprising the step of providing for a user to input 
data within a sterile environment. 

74 . The method of claim 53 wherein the step of acquiring the plurality of images 
comprises the step of acquiring any of a predetermined number of image types. 
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75. The method of claim 74 wherein the step of acquiring the plurality of images from 
the separate image sources comprises the step of acquiring different types of images 
from a single type of imaging source . 

76. The method of claim 53 further comprising manipulating both of the images 
independently of each other. 

77. The method of claim 53 wherein the step of manipulating the at least two images 
comprises shaping and sizing at least one of the at least two images. 

78. The method of claim 77 wherein the step of manipulating comprises shaping and 
sizing at least one of the images to conform to at least one other image in shape and 
size . 

79. The method of claim 53 further comprising superimposing at least one of the at 
least two images upon the other of the at least two images . 

80. The method of claim 53 wherein the step of manipulating the at least two images 
comprises utilizing at least one procedure selected from the group consisting of 
filtering, smoothing, sharpening, pseudocoloring, and edge detection. 

81. The method of claim 80 wherein the edge detection procedure comprises at least one 
routine selected from the group consisting of Laplacian, Roberts, Sorbel, and Frei 
routines . 

82. The method of claim 53 further comprising recording and archiving the method. 

83. The method of claim 53 further comprising providing stereotactic coordinates for a 
user selected point on the presentation. 

84 . The method of claim 53 further comprising the step of simulating at least one 
image . 

85. The method of claim 84 wherein the step of simulating at least one image comprises 
the step of simulating at least one acquired image. 

86. The method of claim 84 wherein the step comprising simulating at least one image is 
selected from the group consisting of simulating at least one two-dimensional graphic 
image and simulating at least one three-dimensional graphic image. 

87. The method of claim 84 wherein the step of simulating at least one image comprises 
the step of simulating a composite of images from more than one imaging source. 

88. The method of claim 87 wherein the step of simulating at least one image comprises 
the step of simulating a composite of manipulated images. 

89. The method of claim 88 further comprising the step of independently manipulating 
components of the composite image as subsets. 

90. The method of claim 89 further comprising the step of independently manipulating 
the subsets independently of the other images. 

91. The method of claim 53 wherein the displaying of images comprises displaying a 
composite of images. 

92. The method of claim 91 wherein the step of displaying a composite of images 
comprises the step of displaying at least one acquired image and one simulated image. 

93. The method of claim 92 further comprising the step of independently manipulating 
the acquired image and the simulated image. 

94. The method of claim 53 further comprising the step of utilizing an infrared grid 
disposed across the image displaying device . 

95. The method of claim 53 further comprising simulating at least one device selected 
from the group consisting of stereotactic devices, probes, electrodes, implantation 
devices, and needles. 
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96. The method of claim 95 further comprising simulating manipulation of the device. 

97. The method of claim 95 wherein the device comprises an isodose implantation device 
and the method further comprises determining optimum placement for isodose 
implantation, determining optimum dosage for isodose implantation and simulating an 
isodose implant. 

98. The method of claim 53 wherein the step of determining stereotactic coordinates 
further comprises determining stereotactic coordinates of at least one entry point, 
target point, placement point and trajectory of a device. 

99. The method of claim 53 further comprising providing pixel analysis. 

100. The method of claim 99 wherein the step of providing pixel analysis is selected 
from the group consisting of providing pixel analysis to an entire image, a 
user-defined area of an image and a combination thereof. 

101. The method of claim 100 further comprising means for selecting image data on the 
images on the basis of the pixel analysis. 

102. The method of claim 101 further comprising the step of generating composite images 
from the image data so as to provide at least one representation selected from the 
group consisting of anatomical configurations, physiological configurations, 
electrophysiological configurations, and combinations thereof, of a patient's brain. 

103. The method of claim 53 wherein step of generating a presentation of the images 
comprises the step of generating a video presentation. 

104. The method of claim 53 wherein the step of generating a presentation of images is 
independent of the step of acquiring a plurality of images from a plurality of separate 
imaging sources . 

105. The method of claim 53 further comprising coding and storing representations of 
physiological response points and selectively recalling and displaying any of the 
response points within the presentation. 

106. The method of claim 53 further comprising the steps of: 

a) providing an image acquiring apparatus; 

b) providing a stereotactic reference system to a patient; 

c) placing the patient's body part to be analyzed in the imaging source; 

d) acquiring the images desired using the image source; 

e) saving and storing the acquired image in the apparatus; 

f ) taking the apparatus and the patient into the operating suite; 

g) using image manipulation routines in the apparatus to calibrate the desired image 
areas; 

h) using image manipulation routines in the apparatus to enhance desired image areas; 
and 

i) using the apparatus to determine desired coordinates. 

107. The method of claim 106 further comprising using a brain map to precisely define 
the desired coordinates. 

108. The method of claim 106 further comprising determining volumetric regions of the 
brain . 

109. The method of claim 106 further comprising simulating brain surgical procedures. 
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L2: Entry 26 of 53 File: USPT Dec 28, 1993 

US- PAT-NO: 5274759 

DOCUMENT- IDENTIFIER: US 5274759 A 

TITLE: Ultrasonic diagnostic apparatus capable of mult i- frame representation 
DATE- ISSUED: December 28, 1993 
INVENTOR- INFORMATION : 

NAME CITY STATE ZIP CODE COUNTRY 

Yoshioka; Yoshihisa Tochigi JPX 

US -CL- CURRENT: 600/440; 345/660, 345 /781, 345/961 
CLAIMS : 

What is claimed is: 

1. An image display apparatus comprising: 

first memory means for sequentially storing a plurality of image data of an object; 

multi-frame buffer memory means for temporarily storing said plurality of image data; 

image selecting means for selecting a plurality of selected image data desired by a 
user from said plurality of image data; 

multi- frame image producing means for sequentially storing said plurality of image data 
from said multi-frame buffer memory means, and for producing multi-frame image data 
constructed of said plurality of selected image data from the multi-frame buffer memory 
means; and 

display means having a display screen for selectively displaying one of a plurality of 
images of the object with a full-screen size in response to said plurality of image 
data from the multi-frame image producing means, and a multi-frame image of the object 
with a reduced- screen size corresponding to each of the selected image data; 

the image selecting means being used to select said plurality of selected image data 
while a plurality of images of said object in full-screen size are sequentially 
displayed on said display screen. 

2. An image display apparatus as claimed in claim 1, wherein a total number of said 
selected image data is selected to be 4 , whereby said multi-frame image of the object 
is constructed of 4 selected images, each of which is displayed with a l/4-reduced 
screen size of the screen. 

3. An image display apparatus as claimed in claim 1, wherein a memory capacity of said 
first memory means is larger than that of said multi- frame image producing means. 

4. An ultrasonic diagnostic apparatus comprising: 

ultrasonic image data acquiring means for acquiring ultrasonic image data of an object 
to produce* ultrasonic tomographic image data; 

first memory means for sequentially storing a plurality of ultrasonic tomographic image 
data; 
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multi-frame buffer memory means for temporarily storing said plurality of ultrasonic 
tomographic image data; 

tomographic image selecting means for selecting a plurality of selected tomographic 
image data desired by a user from said plurality of ultrasonic image data; 

multi- frame image producing means for sequentially storing said plurality of ultrasonic 
tomographic image data from said multi-frame buffer memory means, and for producing 
multi-frame tomographic image data constructed of said plurality of selected 
tomographic image data from the multi-frame buffer memory means; and 

display means having a display screen for selectively displaying one of a plurality of 
ultrasonic tomographic images of said object with a full-screen size in response to 
said plurality of ultrasonic tomographic image data from the multi- frame image 
producing means, and a multi-frame ultrasonic tomographic image of the object with a 
reduced- screen size corresponding to each of the selected tomographic image data; 

the tomographic image selecting means being used to select said plurality of selected 
tomographic image data while a plurality of tomographic images of said object in 
full-screen size are sequentially displayed on said display screen. 

5. An ultrasonic diagnostic apparatus as claimed in claim 4, wherein said ultrasonic 
image data acquiring means includes: 

an ultrasonic transducer for transmitting ultrasonic waves to the object under 
examination and for receiving ultrasonic echoes reflected from the object to produce 
ultrasonic echo signals; and, 

a transmitter/receiver circuit for transmitting an excitation pulse to the ultrasonic 
transducer so as to produce the ultrasonic waves and for receiving the ultrasonic echo 
signals thereto to produce the ultrasonic tomographic image data. 

6. An ultrasonic diagnostic apparatus as claimed in claim 4, wherein said first memory 
means includes: 

a storage unit for sequentially storing said plurality of ultrasonic tomographic image 
data; 

a storage address generator for generating read/write addresses of the storage unit; 
and, 

a storage read/write changing circuit for changing read/write operations of the storage 
unit by utilizing the read/write addresses. 

7. An ultrasonic diagnostic apparatus as claimed in claim 4, wherein said multi-frame 
buffer memory means includes: 

a multi -frame buffer memory circuit having at least a first memory region for 
temporarily storing said plurality of ultrasonic tomographic image data derived from 
the first memory means, and also a second memory region for sequentially fetching said 
plurality of ultrasonic tomographic image data selected by the tomographic image 
selecting means. 

8. An ultrasonic diagnostic apparatus as claimed in claim 7, wherein said multi-frame 
buffer memory means further includes: 

a buffer memory address generator for generating read/write addresses of the 
multi-frame buffer memory circuit; and, 

a buffer memory read/write changing circuit for changing read/write operations of both 
said first and second memory regions by utilizing the read/write addresses. 

9. An ultrasonic diagnostic apparatus as claimed in claim 4, wherein a memory capacity 
of said first memory means is selected to be greater than that of said multi- frame 
image producing means. 

10. An ultrasonic diagnostic apparatus as claimed in claim 4, wherein a total number of 
said selected tomographic image data is selected to be 4, whereby said multi- frame 
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v • ~f »•»,• nhieet is constructed of 4 selected tomographic images, each 

tomogr.ptuc ™^!/^f^,^^r~.n -I., of the di.pHy «r-n. 
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What is claimed is: 



1. An ultrasound image processing apparatus, 



comprising: 



ultrasound beams; 

sSd^^S'ssss sr^r-hL. o£ ..id . 

object formed from the tomographic image data, 
three-dimensional transparent image of the object. 

3. The ultrasound image processing aPP^^^^ ^aid 

4 . The ultrasound image Pressing ^^^S^SU^i'iirit modfin which said 
display means is constructed so as to switchaB displayed simultaneously, a 

^ d ^iri^r?^=^ ^ is displayed ' and a third mode in 

which said tomographic image is displayed. 
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echo data obtained from the sectional plane. 

,. T„e uUr.sound P~— in. " U"^"^'^ 

display means. 

point of the range along the ultrasound beam direction, 
of tS range along the ultrasound beam direction. 

L-L? ^^^^^-----^ 

the range determined by said range determining means. 

12. The ultrasound image processing apparatus as claimed in claim 8 ^» t £% th 
ofthf indingToint y o^S f^T^lTe detained by Lid range determining 

means . 

13 An ultrasound image processing apparatus, comprising: 

of the ultrasound beams; 

,c). to„ g „phio i„g. producing «•» «« gfSf ^cnoXS^H^S.aV.n. 
^^^^ ^ ^ 

displayed three-dimensional image . 

15 An ultrasound image processing apparatus, comprising: 

ultrasound beams; 

(c, three-dimensional image producing means for P-^^^^^^f ^S^d^a ootainel 

(d) means for simultaneously displaying both said three-dimensional image and said 
tomographic im a g* on a display in a contrast manner. 



2 of 3 



4/30/02 2:59 PM 



J " , " h ^-//w e stbrs:8002^in/gatexxe?f^oc&s...ge=&p_Message=&p_doicnt=l&p_doc_l-Kl hCLM 

Record Display Form 1 r- 

Tfi The ultrasound image processing apparatus as claimed in claim 15. wherein the range 
naving the echo dtta from^hich safd ?nree-dimensional image xs to be produced is 
indicated on said displayed tomographic image. 

17. A method of forming and displaying ultrasound images, comprising the steps of: 

(a) transmitting ultrasound beams toward an object in a three-dimensional region and 

receiving echoes of the ultrasound beams; 

ultrasound beams; 

(c) producing a tomographic image data of said object based on a part of the echo data 
of the received ultrasound beams; and 

(d) displaying a three-dimensional image which is formed based on the th ^ ee -f^ s ^f 
Jmiae data and a ^nn r-^Ma image which is formed based on the tomo graphic image data 
orfstnale displ ar simu?taneous?y such that thes e images are contrasted with each 
other . 

18. The method of forming and displaying ultrasound i"****" ™ fdesired 
further comprising a step of setting a sectional plane of said object at a desired 
poSSon on said displayed three-dimensional image, wherein said tomographic image is 
formed from the echo data obtained from the sectional plane. 

19. The method of forming and displaying ultrasound images as claimed in e 1 *™ 11 ' 
further comprising a step of determining a range containing echo data to be processed 
baS on said displayed tomographic image, wherein said three-dimensional image is 
produced from the echo data obtained from said range. 

20. A method of diagnosing a living body using ultrasound images, comprising the steps 



of: 



(a) transmitting ultrasound beams toward a three-dimensional region of a living body 
and receiving echoes of the transmitted ultrasound beams; 

(b) producing a three-dimensional image data of an object in « aid ^^ e -f^ n " si ° nal 
region in a real time base based on echo data obtained from the echoes of the 

ultrasound beams; 

(c) producing a B-mode image along a sectional plane of the object based on a part of 

the echo data; and 

(d) displaying said three-dimensional image and said B-mode image «i^ ta " e ° u f * on a 
single screen in a contrast manner, thereby diagnosing the object based on the 
displayed images. 
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